Born in Chicago Heights, USA, in 1923, Pate learned to play piano and tuba as a child. Later while serving in the Army, he picked up the bass and learned arranging. He had stints with Coleridge Davis and Stuff Smith in the '40s. During the '50s, he was the resident arranger for the in-house band of Club Delisa. Recording for Federal as the Johnny Pate Quintet, the bandleader/arranger/bassist had a hit with "Swinging Shepherd Blues," which hit number 17 R&B on Billboard's charts in spring 1958.